2 Amazon Simple Storage Service (AWS S3)

Plugin adds the s3upload method to the common io.File class, you can call this method passing a simple closure configuring the upload request or even with no parameters to upload the file following the application config.

All these properties will be described in the next topic of this guide.

Using default plugin configuration

def s3file = new File("test.txt").s3upload()

Configuring per upload

def s3file = new File("test.txt").s3upload {
   bucket "xxx"
   path "test/files/"
   //more config options
}

2.1 Specific configuration for S3

In your config.groovy, inside grails.plugin.aws closure, you can set some extra configurations for S3 usage. This properties will be used when no respective config is defined when uploading file.

For example, you can define a default bucket for file uploads. Everytime you attemp to upload a file without explicitly setting the bucket name, this default will be used.

Check now, all default config possibilities you can set.

Bucket name

To set a default bucket, that will be used to all file upload use the config below

grails {
   plugin {
      aws {
         s3 {
            bucket = "grails-plugin-test"
         }
      }
   }
}

During the first upload on this bucket, it will be created if does not exist.

ACL (file permission)

The permissions that will be granted on this file, you can use:

To configure public access as default to all file uploads, use this:

grails {
   plugin {
      aws {
         s3 {
            acl = "public"
         }
      }
   }
}

If you like to set private access to your files, you should config this way:

grails {
   plugin {
      aws {
         s3 {
            acl = "private"
         }
      }
   }
}

RRS - Reduced Redundancy Storage

RRS stored files provides a cheaper storage with 99.99% durability instead of 99.999999999% as the default provided by AWS S3. More information here: http://aws.amazon.com/about-aws/whats-new/2010/05/19/announcing-amazon-s3-reduced-redundancy-storage/

This is disabled by default, if you like to set RRS enabled for all uploads, use this config key:

grails {
   plugin {
      aws {
         s3 {
            rrs = true
         }
      }
   }
}

2.2 Uploading files

As described in the beggining, the plugin adds a s3upload(...) method to File class, so you'll just need to call this method passing a closure with overwritten config options and other ones, if you do not want to overwrite, just leave this and the plugin will catch from the Config.groovy default options.

The s3upload operation returns an instance of grails.plugin.aws.s3.S3File. As this plugin uses jets3t (http://jets3t.s3.amazonaws.com/index.html) to handle file upload, the S3File is just a wrapper for a delegated jets3t S3Object instance as you can see below:

package grails.plugin.aws.s3
import org.jets3t.service.model.S3Object

class S3File {

@Delegate S3Object source

public S3File(S3Object _source) { this.source = _source } }

So, you can call any S3Object method on S3File instance. S3Object API is available here: http://jets3t.s3.amazonaws.com/api/org/jets3t/service/model/S3Object.html, for example, to retrieve the ETag hash for the S3File uploaded you would just call:

def s3file = … //upload the file
def etag = s3file.getETag()

Simple File upload

def s3file = new File("/tmp/test.txt").s3upload {
	path "folder/to/my/file/"
}

This way your test.txt file will be uploaded to

<default-bucket>.s3.amazonaws.com/folder/to/my/file/test.txt

If you want to overwrite config during the file upload, check this guide next section.

Storing file information for later retrieval

If you are uploading some picture to S3, you'll probably need to store information on how to get that file again later. The S3File object returned will give all information you'll need. Now, depends on what information you want to store. Follow S3Object docs (link above) and get whatever you want.

A common approach would be storing the bucket, path and file (key). With these parameters you can rebuild the URL to reach the file, or even build another S3Object to delete it. Some next version of this plugin will allow s3file deletion.

2.2.1 Setting file virual path

S3 does not support paths or buckets inside other buckets, to solve this and keep your files organized, you can use the path method inside the config closure. Doing this, the plugin will set a metadata into this file telling AWS that this file is virtually in a folder that does not exist.

The effect is exactly like in a regular folder. For example, doing the upload below:

def uploadedFile = new File("/tmp/profile-picture.jpg").s3upload {
	bucket "my-aws-app"
	path "pictures/user/profile/"
}

The file will be stored and available in the following url:

http://my-aws-app.s3.amazonaws.com/pictures/user/profile/profile-picture.jpg

And using the AWS S3 console, the files will visually be inside folders either. Some third-party apps is already using this feature to show "folders".

2.2.2 Overwriting AWS credentials

Just call the credentials method inside the upload closure, and this credentials will be used (for this upload only). Example:

def uploadedFile = new File("/tmp/test.txt").s3upload {
	credentials "my-other-access-key", "my-other-secret-key"
}

2.2.3 Overwriting bucket to file upload

You can call the bucket method and define witch different bucket (from default) will be used. This bucket will be created if does not exist.

def uploadedFile = new File("/tmp/test.txt").s3upload {
	bucket "other-bucket"
}

This file will be uploaded to

other-bucket.s3.amazonaws.com/test.txt

Remember, when plugin created a non pre-existent bucket, it will be created in the default US region. If you like to set a different location, just pass a second string parameter containing the region string. For example, to set this bucket creation in Europe region:

def uploadedFile = new File("/tmp/test.txt").s3upload {
	bucket "bucket-not-yet-created-in-europe", "EU"
}

2.2.4 ACL (file permission)

The permissions that will be granted on this file, you can use the same values shown in "General Plugin Config" topic on this guide.

def uploadedFile = new File("/tmp/test.txt").s3upload {
	acl "private"
}

2.2.5 RRS - Reduced Redundancy Storage

If some specifically file you like to use a different RRS setting, call the rrs method in the closure, passing true or false, as you wish

def uploadedFile = new File("/tmp/test.txt").s3upload {
	rrs false
}

2.2.6 Setting File Metadata

AWS S3 files can store user metadata, doing this is simple as setting a metadata map to file upload

def uploadedFile = new File("/tmp/test.txt").s3upload {
	metadata [user-id: 123, username: 'johndoe', registered-date: new Date().format('dd/MM/yyyy')]
}

2.2.7 Creating public URLs for private files

When you upload a file to S3 with a "private" acl, means that the file won't be accessed directly using the URL, but only with your amazon credentials.

For example:

def s3file = new File("test.txt").s3upload {
   bucket "secret-files"
   acl "private"
}

This will make the file above (http://secret-files.s3.amazonaws.com/test.txt) inaccessible using its URL. If you need to allow someone to get the file in a short period of time, you can ask the plugin to generate a public URL for it. The object will remain private, but you can use the returned URL to access the file in a short period of time. For example:

def s3file = new File("test.txt").s3upload {
   bucket "secret-files"
   acl "private"
}

def publicUrl = s3file.publicUrlFor()

So, the value on the publicUrl is the URL for accessing the file. By default the URL will be valid for 1 hour. Every requests on it will return the uploaded object. After one hour, next requests on it will return an error: "Access Denied for Object".

Defining when public URL will expire

You can set the expires date for the public URL, passing an argument to publicUrlFor method.

s3file.publicUrlFor(3.hours) //will be available for 3 hours
s3file.publicUrlFor(10.years) //available for 10 years
s3file.publicUrlFor(1.second) //you won't get this one on time

You can any of these methods:

1.second or 2.seconds 
1.minute or 2.minutes
1.hour or 2.hours
1.day or 2.days
1.month or 2.months
1.year or 2.years

This properties are injected on Integer class, so, enjoy the magic.

2.2.8 Creating torrent for S3 hosted files

It is possible to generate torrent URLs for S3 hosted files with the plugin.

After uploading some file, just call the torrent() method on it and you'll get the torrent URL for it.

def s3file = new File("test.txt").s3upload {
   bucket "secret-files"
   acl "private"
}

def torrentUrl = s3file.torrent()

From the AWS docs:

There is no extra charge for use of BitTorrent with S3. Data transfer via the BitTorrent protocol is metered at the same rate as client/server delivery. To be precise, whenever a downloading BitTorrent client requests a “piece” of an object from the S3 “seeder”, charges accrue just as if an anonymous request for that piece had been made using the REST or SOAP protocol. These charges will appear on your S3 bill and usage reports in the same way. The difference is that if a lot of clients are requesting the same object simultaneously via BitTorrent, then the amount of data S3 must serve to satisfy those clients will be lower than with client/server delivery. This is because the BitTorrent clients are simultaneously uploading and downloading amongst themselves. The data transfer savings achieved from use of BitTorrent can vary widely depending on how popular your object is. Less popular objects require heavier use of the “seeder” to serve clients, and thus the difference between BitTorrent distribution costs and client/server distribution costs may be small for such objects. In particular, if only one client is ever downloading a particular object at a time, the cost of BitTorrent delivery will be the same as direct download.